On Combining Reinforcement Learning and Monte Carlo for Dynamic Virtual Network Embedding

Demo paperInternational audienceNetwork slicing is one of the key building blocks in the evolution towards "zero touch networks". Indeed, this will allow 5G and beyond 5G networks to deploy services dynamically, on the same substrate network, regardless of their constraints. In this demo, we introduced a platform for dynamic virtual network embedding, a problem class known to be NP-hard. The proposed solution is based on a combination of a deep reinforcement learning strategy and a Monte Carlo (MC) approach. The idea here is to learn to generate, using a Deep Q-Network (DQN), a distribution of the placement solution, on which a MC-based search technique is applied. This makes the agent's exploration of the solution space more efficient

A Robust Monte-Carlo-Based Deep Learning Strategy for Virtual Network Embedding

International audienceNetwork slicing is one of the building blocks in Zero Touch Networks. It mainly consists in a dynamic deployment of services in a substrate network. However, the Virtual Network Embedding (VNE) algorithms used generally follow a static mechanism, which results in sub-optimal embedding strategies and less robust decisions. Some reinforcement learning algorithms have been conceived for a dynamic decision, while being time-costly. In this paper, we propose a combination of deep Q-Network and a Monte Carlo (MC) approach. The idea is to learn, using DQN, a distribution of the placement solution, on which a MC-based search technique is applied. This improves the solution space exploration, and achieves a faster convergence of the placement decision, and thus a safer learning. The obtained results show that DQN with only 8 MC iterations achieves up to 44% improvement compared with a baseline First-Fit strategy, and up to 15% compared to a MC strategy

Artificial Intelligence for Elastic Management and Orchestration of 5G Networks

The emergence of 5G enables a broad set of diversified and heterogeneous services with complex and potentially conflicting demands. For networks to be able to satisfy those needs, a flexible, adaptable, and programmable architecture based on network slicing is being proposed. A softwarization and cloudification of the communications networks is required, where network functions (NFs) are being transformed from programs running on dedicated hardware platforms to programs running over a shared pool of computational and communication resources. This architectural framework allows the introduction of resource elasticity as a key means to make an efficient use of the computational resources of 5G systems, but adds challenges related to resource sharing and efficiency. In this article, we propose Artificial Intelligence (AI) as a built-in architectural feature that allows the exploitation of the resource elasticity of a 5G network. Building on the work of the recently formed Experiential Network Intelligence (ENI) industry specification group of the European Telecommunications Standards Institute (ETSI) to embed an AI engine in the network, we describe a novel taxonomy for learning mechanisms that target exploiting the elasticity of the network as well as three different resource elastic use cases leveraging AI. This work describes the basis of a use case recently approved at ETSI ENI.Part of this work has been performed within the 5G-MoNArch project (Grant Agreement No. 761445), part of the Phase II of the 5th Generation Public Private Partnership (5G-PPP) program partially funded by the European Commission within the Horizon 2020 Framework Program. This work was also supported by the the 5G-Transformer project (Grant Agreement No. 761536)

Le multihoming dans les réseaux sans fil hétérogènes

Fifth generation mobile networks (5G) are being designed to introduce new services that require extreme broadband data rates and utlra-reliable latency. 5G will be a paradigm shift that includes heterogeneous networks with densification, virtualized radio access networks, mm-wave carrier frequencies, and very high device densities. However, unlike the previous generations, it will be a holistic network, tying any new 5G air interface and spectrum with the currently existing LTE and WiFi. In this context, we focus on new resource allocation strategies that are able to take advantage of multihoming in dual access settings. We model such algorithms at the flow level and analyze their performance in terms of flow throughput, system stability and fairness between different classes of users. We first focus on multihoming in LTE/WiFi heterogeneous networks. We consider network centric allocations where a central scheduler performs local and global proportional fairness (PF) allocations for different classes of users, single-homed and multihomed users. By comparison with a reference model without multihoming, we show that both strategies improve system performance and stability, at the expense of more complexity for the global PF. We also investigate user centric allocation strategies where multihomed users decide the split of a file using either peak rate maximization or network assisted strategy. We show that the latter strategy maximizes the average throughput in the whole network. We also show that network centric strategies achieve higher data rates than the user centric ones. Then, we focus on Virtual Radio Access Networks (V-RAN) and particularly on multi-resource allocation therein. We investigate the feasibility of virtualization without decreasing neither users performance, nor system's stability. We consider a 5G heterogeneous network composed of LTE and mm-wave cells in order to study how high frequency networks can increase system's capacity. We show that network virtualization is feasible without performance loss when using the dominant resource fairness strategy (DRF). We propose a two-phase allocation (TPA) strategy which achieves a higher fairness index than DRF and a higher system stability than PF. We also show significant gains brought by mm-wave instead of WiFi. Eventually, we consider energy efficiency and compare DRF and TPA strategies with a Dinklebach based energy efficient strategy. Our results show that the energy efficient strategy slightly outperforms DRF and TPA at low to medium load in terms of higher average throughput with comparable power consumption, while it outperforms them at high load in terms of power consumption. In this case of high load, DRF outperforms TPA and the energy efficient strategy in terms of average throughput. As for Jain's fairness index, TPA achieves the highest oneLes réseaux mobiles de la cinquième génération (5G) sont conçus pour introduire de nouveaux services nécessitant des débits de données extrêmement hauts et une faible latence. 5G sera un changement de paradigme qui comprend des réseaux hétérogènes densifiés, des réseaux d'accès radio virtualisés, des fréquences porteuses à ondes millimétrées et des densités de périphériques très élevées. Cependant, contrairement aux générations précédentes, 5G sera un réseau holistique, intégrant n'importe quelle nouvelle technologie radio avec les technologies LTE et WiFi existant. Dans ce contexte, on se concentre sur de nouvelles stratégies d'allocation de ressources capables de bénéficier du multihoming dans le cas d'accès double au réseau. On modélise ces algorithmes au niveau du flux et analyse leurs performances en termes de débit, de stabilité du système et d'équité entre différentes catégories d'utilisateurs. On se concentre tout d'abord sur le multihoming dans les réseaux hétérogènes LTE/WiFi. On considère les allocations centrées sur le réseau où un planificateur central effectue des allocations d'équité proportionnelle (PF) locale et globale pour différentes classes d'utilisateurs, utilisateurs individuels (single-homed) et multi-domiciliés (multihomed). Par rapport à un modèle de référence sans multihoming, on montre que les deux stratégies améliorent la performance et la stabilité du système, au détriment d'une plus grande complexité pour la stratégie PF globale. On étudie également les stratégies d'allocation centrées sur l'utilisateur, dans lesquelles les utilisateurs multihomed décident la partition de la demande d'un fichier en utilisant soit la maximisation du débit crête, soit la stratégie assistée par réseau. On montre que cette dernière stratégie maximise le débit moyen dans l'ensemble du réseau. On montre également que les stratégies centrées sur le réseau permettent d'obtenir des débits de données plus élevés que ceux centrés sur l'utilisateur. Ensuite, on se concentre sur les réseaux d'accès radio virtuels (V-RAN) et en particulier sur l'allocation de multi-ressources. On étudie la faisabilité de la virtualisation sans diminuer ni la performance des utilisateurs, ni la stabilité du système. On considère un réseau hétérogène 5G composé de cellules LTE et mm-wave afin d'étudier comment les réseaux hauts fréquence peuvent augmenter la capacité du système. On montre que la virtualisation du réseau est réalisable sans perte de performance lors de l'utilisation de la stratégie « dominant resource fairness » (DRF). On propose une stratégie d'allocation en deux phases (TPA) qui montre un indice d'équité plus élevé que DRF et une stabilité du système plus élevée que PF. On montre également des gains importants apportés par l'adoption des fréquences mm-wave au lieu de WiFi. Finalement, on considère l'efficacité énergétique et compare les stratégies DRF et TPA avec une stratégie éconergétique basée sur l'algorithme de Dinklebach. Les résultats montrent que la stratégie éconergétique dépasse légèrement DRF et TPA à charge faible ou moyenne en termes de débit moyen plus élevé avec une consommation d'énergie comparable, alors qu'elle les surpasse à une charge élevée en termes de consommation d'énergie moins élevée. Dans ce cas de charge élevée, DRF surpasse TPA et la stratégie éconergétique en termes de débit moyen. En ce qui concerne l'indice d'équité de Jain, TPA réalise l'indice d'équité le plus élevé parmi d'autres stratégie

A Frequency-Based Intelligent Slicing in LoRaWAN with Admission Control Aspects

International audienceThe significant deployment of LoRaWan networks is increasingly questioning its ability to handle massive numbers of IoT devices and its ability to support service differentiation. The few existing attempts to implement service differentiation suffer from a lack of scalability and do not meet the qualitative criteria of the services, since without admission control there is no way to restrain the devices from transmitting. In this paper, we present a scalable probabilistic approach that not only enables an efficient sharing of LoRaWan access networks between different services/slices, but more importantly allows achieving the objectives of the supported services through the integration of an admission control. Since the derivation of devices' repartition probabilities is a very complex problem, we propose an evolutionary algorithm to derive them efficiently. The obtained results clearly show the ability of the proposed solution to efficiently utilize the scarce radio resources, while achieving the qualitative objectives of the prioritized services

On Combining Reinforcement Learning and Monte Carlo for Dynamic Virtual Network Embedding

Demo paperInternational audienceNetwork slicing is one of the key building blocks in the evolution towards "zero touch networks". Indeed, this will allow 5G and beyond 5G networks to deploy services dynamically, on the same substrate network, regardless of their constraints. In this demo, we introduced a platform for dynamic virtual network embedding, a problem class known to be NP-hard. The proposed solution is based on a combination of a deep reinforcement learning strategy and a Monte Carlo (MC) approach. The idea here is to learn to generate, using a Deep Q-Network (DQN), a distribution of the placement solution, on which a MC-based search technique is applied. This makes the agent's exploration of the solution space more efficient

On Combining Reinforcement Learning and Monte Carlo for Dynamic Virtual Network Embedding

Demo paperInternational audienceNetwork slicing is one of the key building blocks in the evolution towards "zero touch networks". Indeed, this will allow 5G and beyond 5G networks to deploy services dynamically, on the same substrate network, regardless of their constraints. In this demo, we introduced a platform for dynamic virtual network embedding, a problem class known to be NP-hard. The proposed solution is based on a combination of a deep reinforcement learning strategy and a Monte Carlo (MC) approach. The idea here is to learn to generate, using a Deep Q-Network (DQN), a distribution of the placement solution, on which a MC-based search technique is applied. This makes the agent's exploration of the solution space more efficient

A Robust Monte-Carlo-Based Deep Learning Strategy for Virtual Network Embedding

International audienceNetwork slicing is one of the building blocks in Zero Touch Networks. It mainly consists in a dynamic deployment of services in a substrate network. However, the Virtual Network Embedding (VNE) algorithms used generally follow a static mechanism, which results in sub-optimal embedding strategies and less robust decisions. Some reinforcement learning algorithms have been conceived for a dynamic decision, while being time-costly. In this paper, we propose a combination of deep Q-Network and a Monte Carlo (MC) approach. The idea is to learn, using DQN, a distribution of the placement solution, on which a MC-based search technique is applied. This improves the solution space exploration, and achieves a faster convergence of the placement decision, and thus a safer learning. The obtained results show that DQN with only 8 MC iterations achieves up to 44% improvement compared with a baseline First-Fit strategy, and up to 15% compared to a MC strategy

A Robust Monte-Carlo-Based Deep Learning Strategy for Virtual Network Embedding

International audienceNetwork slicing is one of the building blocks in Zero Touch Networks. It mainly consists in a dynamic deployment of services in a substrate network. However, the Virtual Network Embedding (VNE) algorithms used generally follow a static mechanism, which results in sub-optimal embedding strategies and less robust decisions. Some reinforcement learning algorithms have been conceived for a dynamic decision, while being time-costly. In this paper, we propose a combination of deep Q-Network and a Monte Carlo (MC) approach. The idea is to learn, using DQN, a distribution of the placement solution, on which a MC-based search technique is applied. This improves the solution space exploration, and achieves a faster convergence of the placement decision, and thus a safer learning. The obtained results show that DQN with only 8 MC iterations achieves up to 44% improvement compared with a baseline First-Fit strategy, and up to 15% compared to a MC strategy